…-abc

- Improve `Engines` performance and implementation

for more information, see https://pre-commit.ci

for more information, see https://pre-commit.ci

for more information, see https://pre-commit.ci

# Conflicts:
#	tiatoolbox/utils/misc.py

# Conflicts:
#	tests/models/test_feature_extractor.py
#	tiatoolbox/models/models_abc.py

# Conflicts:
#	tiatoolbox/cli/common.py
#	tiatoolbox/cli/nucleus_instance_segment.py
#	tiatoolbox/cli/patch_predictor.py
#	tiatoolbox/models/engine/semantic_segmentor.py

* ⚡ Make WSIPatchDataset Pickleable to Support Windows Multithreading (#947)

This PR makes the WSIPatchDataset class picklable by delaying the creation of the reader object until the first call to `__getitem__`. This enables the use of multiple loader workers on Windows without errors and provides significant performance improvements.

- Delays reader object instantiation to the first `__getitem__` call instead of during initialization
- Extracts reader creation logic into a separate `_get_reader` method
- Stores image path and mode as instance variables for lazy initialization

Speedup for the WSI prediction cell of the patch_prediction example notebook: 
2min 48 sec with 0 loader workers -> 1min 13 sec with 4 workers.

Note: this PR doesn't have any effect for Linux as the multi-threading already works fine there because Linux multithreading doesn't require things to be pickleable

* 🔀 Merge branch develop into dev-engine-abc

* 🐛 Fix reader_info read

---------

Co-authored-by: Mark Eastwood <[email protected]>

# Conflicts:
#	tiatoolbox/models/dataset/classification.py

# Conflicts:
#	tests/models/test_patch_predictor.py

for more information, see https://pre-commit.ci

for more information, see https://pre-commit.ci

# Conflicts:
#	tests/models/test_feature_extractor.py
#	tests/models/test_multi_task_segmentor.py
#	tests/models/test_nucleus_instance_segmentor.py
#	tests/models/test_patch_predictor.py
#	tests/models/test_semantic_segmentation.py
#	tiatoolbox/models/architecture/__init__.py

## Summary of Changes

### Major Additions
- **Dask Integration:**  
  - Added `dask` as a dependency and integrated Dask arrays and lazy computation throughout the engine and patch predictor code.
  - Added Dask-based merging, chunking, and memory-aware processing for large images and WSIs.

- **Zarr Output Support:**  
  - Added support for saving model predictions and intermediate results directly to Zarr format.
  - New CLI options and internal logic for Zarr output, including memory thresholding and chunked writes.

- **SemanticSegmentor Engine:**  
  - Added a new `SemanticSegmentor` engine with Dask/Zarr support and new test coverage (`test_semantic_segmentor.py`).
  - Added CLI entrypoint for `semantic_segmentor` and removed the old `semantic_segment` CLI.

- **Enhanced CLI and Config:**  
  - Added CLI options for memory threshold, unified worker options, and improved mask handling.
  - Updated YAML configs and sample data for new models and test images.

- **Utilities and Validation:**  
  - Added utility functions for minimal dtype casting, patch/stride validation, and improved error handling (e.g., `DimensionMismatchError`).
  - Improved annotation store conversion for Dask arrays and Zarr-backed outputs.

- **Changes to `kwarg`**
  - Add `memory-threshold`
  - Unified `num-loader-workers` and `num-postproc-workers` into `num-workers`
  - Removed `cache_mode` as cache mode is automatically handled.

---

### Major Removals/Refactors
- **Removed Old CLI and Redundant Code:**  
  - Deleted the old `semantic_segment.py` CLI and replaced it with `semantic_segmentor.py`.
  - Removed legacy cache mode and patch prediction Zarr store tests.

- **Refactored Model and Dataset APIs:**  
  - Unified and simplified model inference APIs to always return arrays (not dicts) for batch outputs.
  - Refactored dataset classes to enforce patch shape validation and remove legacy “mode” logic.

- **Test Cleanup:**  
  - Removed or updated tests that relied on old APIs or cache mode.
  - Refactored test assertions for new output types and Dask array handling.

- **API Consistency:**  
  - Standardized function and argument names across engines, CLI, and utility modules.
  - Updated docstrings and type hints for clarity and consistency.

---

### Notable File Changes
- **New:**  
  - `tiatoolbox/cli/semantic_segmentor.py`
  - `tests/engines/test_semantic_segmentor.py`

- **Removed:**  
  - `tiatoolbox/cli/semantic_segment.py`
  - Old cache mode and patch Zarr store tests

- **Heavily Modified:**  
  - `engine_abc.py`, `patch_predictor.py`, `semantic_segmentor.py`
  - CLI modules and test suites
  - Dataset and utility modules for Dask/Zarr compatibility

---

### Impact

- Enables scalable, parallel, and memory-efficient inference and output saving for large images.
- Simplifies downstream analysis by supporting Zarr as a native output format.
- Lays the groundwork for further Dask-based optimizations in TIAToolbox.


---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>